Accuracy and Pitfalls in the Smartphone-Based Audiometry Examination

Introduction: Approximately 466 million people suffer from hearing loss worldwide, with Indonesia ranking fourth in Southeast Asia. However, conventional pure-tone audiometry is not yet available in many areas because of its high cost. Numerous available smartphone-based audiometry applications are potential alternative screening tools for hearing loss, especially in Indonesia. This study examined the findings on the validation of smartphone-based audiometry applications to assess hearing functions available in Indonesia. Materials and Methods: Based on the established eligibility criteria, this study was conducted by browsing the relevant literature validating smartphone-based audiometry applications in Indonesia. Relevant study data, such as the author, year, location, implementation procedures, and outcomes, were extracted and summarized. Results: This systematic review found 17 relevant and eligible publications. Of the six applications tested, 5 were found to have good validity, such as uHearTM, Audiogram MobileTM, AudCalTM, Hearing TestTM e-audiologia, and WuliraTM. All smartphone-based audiometry was tested only for the air conduction threshold and was influenced by several factors. Conclusion: Because smartphone-based audiometry is inexpensive, simple, and more accessible than conventional audiometric testing, it can be useful as a screening modality or alternative approach to assess hearing function. Unfortunately, smartphone-based audiometry cannot replace conventional audiometry in diagnosing hearing impairment.


Introduction
Hearing is the ability to recognize sounds (1) and is one of the human sensory systems necessary for humans to communicate with their surrounding environment at all times (2); however, hearing loss is commonly reported in every region and is estimated to affect 466 million people worldwide (5.5% of the world's population), and this number is expected to rise to one in every four people by 2050 (3).
After Sri Lanka, Myanmar, and India, Indonesia ranked fourth in Southeast Asia with the highest rate of hearing loss (4).
The national prevalence rate of hearing loss in Indonesia is 2.6%, with East Nusa Tenggara Province leading the way with a rate of 3.7%, followed by Lampung in the second position with a rate of 3.6% (5).Currently, 34 million children have hearing loss, comprising 60% preventable cases (3).More than 1 billion young people aged 12-35 years are at risk of hearing loss due to recreational exposure to loud noises.Approximately one-third of people aged >65 years have hearing loss, with most cases occurring in the South Asian, Asia Pacific, and sub-Saharan African regions.Hearing loss has far-reaching consequences, can be very harmful, and can also lead to social isolation, with people becoming lonely and frustrated, especially among elderly people (3).Early detection (screening) and prompt treatment are important to avoid hearing loss, which has several negative consequences (6).A hearing function examination, using conventional puretone audiometry (PTA) as the gold standard for assessing hearing function, can be performed to determine whether someone has a hearing loss (7,8).However, although a conventional audiometer is a highly recommended test in hearing function tests because of its accuracy, the use of PTA is still not fully applicable because of substantial challenges in many areas, particularly in lower-middle-income areas (6).Because conventional audiometers are very expensive, these tools are still limited and not widely available in many areas.As a result, access to hearing function tests remains challenging, and the costs incurred by an individual for each examination are also quite high (6).Although the number of conventional audiometers is still limited and not widely available in many regions, various new, innovative, low cost, easy-to-use, and automated technologies have been developed to assess hearing function.One of these is smartphone-based audiometry, which can assess hearing functions.Smartphone application developers have used this technology to develop applications that perform independent hearing screening tests.
With the growing use of smartphones globally, audiometry applications offer a promising avenue for screening hearing loss (6).Smartphone-based audiometric solutions have been proposed as a means of lowering costs and increasing access to hearing function tests (9,10).Other studies have found tangible evidence of the use of smartphone applications for hearing assessment in different populations (10)(11)(12)(13).A randomized controlled trial conducted in Turkey has shown that the use of smartphone-based audiometric tests produces results comparable to that of conventional audiometry (14).Smartphone-based audiometry tests are also widely available in Indonesia and can measure hearing function; however, their scientific validity has not been thoroughly reviewed.Therefore, studies on the validation of smartphone-based audiometry available in Indonesia for assessing hearing function are required with smartphone-based audiometry compared to conventional audiometry's gold standard.

Materials and Methods
This systematic review used the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) to conduct the literature search.Based on the established eligibility, this study was conducted by browsing the relevant literature validating smartphone-based audiometry applications in Indonesia.Smartphone-based audiometry applications reviewed in the literature were checked for availability in Indonesia by searching for the application's name on the most popular smartphone commercial application stores in Indonesia, namely, the Google Play Store (Android) and the Apple App Store (iOS).

Literature Search Method
Literature that validated smartphone-based audiometry applications was searched online using three databases, namely, PubMed, ScienceDirect, and Google Scholar, with the following keywords: (Accuracy OR Accurate OR Valid) AND (Audiometry OR Audiometer) AND (Smartphone OR Android OR iOS).This systematic review was conducted using PRISMA.

Eligibility Criteria
The included literature must meet the inclusion and exclusion criteria.The inclusion criteria are as follows: • Participants: All study participants were aged over 4 years.This study excluded literature that could not be accessed in full text, literature in the form of reviews, and literature that did not use English.

Data Analysis Method
The synthesis of qualitative data using a metaaggregation approach was used as the data analysis method in this study.All data from the literature included in accordance with PRISMA were collected using data collection standards established by us.Relevant data of the study, such as the author, year, location, implementation procedures, and outcomes, were extracted and summarized.

Results
A total of 1,107 publications were identified by searching three literature databases: PubMed, ScienceDirect, and Google Scholar.The literature was then systematically examined using the PRISMA stage (Fig. 1), with 85 of them being excluded due to duplicates.The remaining titles and abstracts were reviewed for relevance, and 998 were determined not to meet the inclusion criteria.The relevant literature was reviewed for eligibility, and 17 papers were eventually selected to be included in this study (7,(14)(15)(16)(17)(18)(19)(20)(21)(22)(23)(24)(25)(26)(27)(28)(29).A total of 17 papers included in the collection discussed the validation of smartphone-based audiometry applications available in Indonesia.This systematic review tested six applications for validity in (Table 1).uHear TM , which has been validated by eight papers, is the most validated application for the iOS operating system.Other validated iOS applications include Audiogram Mobile TM , which was validated in two papers, and AudCal TM , which was validated in one paper.Not only iOS-based audiometry applications but also smartphonebased audiometry applications with an Android system were also found, including four papers validating the Hearing Test TM e-audiologic application and one article validating each, the Hearing Test Pro TM e-audiologic and Wulira TM (7,(14)(15)(16)(17)(18)(19)(20)(21)(22)(23)(24)(25)(26)(27)(28)(29).All of the literature included in this survey was a primary study conducted over several years.In 2012, one research article was found; in 2015 and 2017, two papers were found.In 2016, 2018, and 2020, four publications were found.The research for the included publications was also conducted in different countries.Nigeria and Pakistan most commonly conduct validation research on audiometry applications, with several other countries, namely, South Africa, USA, Belgium, Israel, Canada, Malaysia, Oman, Poland, Spain, Taiwan, Turkey, New Zealand, and Uganda.All included literature was in the form of an experimental in-subject design with a prospective, cross-sectional study.The number of patients included in each study varied.The smallest sample size was 20 people, and the largest was 200 people, with a total sample size of 1,455 people drawn from the entire literature (2,910 ears).Almost 60% of the participants were males, ranging from 8 to 91 years (7,(14)(15)(16)(17)(18)(19)(20)(21)(22)(23)(24)(25)(26)(27)(28)(29).As shown in Table 1, the uHear TM application has generally good validity (7,(15)(16)(17)(18), with only three studies finding poor validity (17,19,20).Due to the noisy test environment, an uncelebrated tool, and a 16-bit digital-to-analog converter in iPod/iPhone devices, the uHear TM application's dynamic range is limited to approximately 85 (15-100) dB (19).Because elderly participants could not operate smartphones properly, the validity of uHear TM deteriorated (17).The noisy test environment and difficulties understanding the application instruction language were the study's limitations that resulted in poor validity of the obtained results (20).The uHear TM application is beneficial for screening moderate to severe hearing loss (>40 dB) and is more accurate at higher frequencies (i.e., 2000, 4000, 6000, and 8,000 Hz frequencies).Conversely, the validity of low-frequency measurements was found to be poor (7,16,17,19,20).This is due to environmental noise and occlusion caused by poor transducers.Uncontrolled test environment noise and inefficient use of transducers cause sound in the test environment to enter the ear canal and interfere with puretone transmission, causing a decreased sound pressure of the pure-tone in the ear canal, which in turn vibrates the tympanic membrane and auditory bones.This is more contrasted at low frequencies (250-1,000 Hz) because the number of sound waves produced at low frequencies is less than that produced at high frequencies; therefore, low-frequency pressure in the ear canal significantly increases the hearing threshold because it vibrates less than at high frequencies.The validity of the uHear application has also been proven to improve when performed in a low-noise environment (7,15,19).Audiogram Mobile TM is another application included in this systematic review.This application is beneficial for screening mild to severe hearing impairment.The Audiogram Mobile TM application has been validated in two publications (21,22).The Audiogram Mobile TM application has good validity in screening for mild to severe hearing loss (>20 dB) when used in a low-noise environment.However, it has not achieved as much as uHear TM because of noise background and cut-off threshold-based (22).
Conversely, the Mobile TM Audiogram had low validity but high reliability.Poor validity is due to the type of transducer used, which did not occlude the ear, leading to some tone leakage from the ear canal, resulting in a lower overall sound pressure level in the ear canal.Another reason for participant distraction during application testing was that the examiner was in a soundproof booth with the patient.When the tester is present, the participant may find it more distracting because they should pay attention to many factors, including the sound of the tester operating the iPad.Despite some limitations in its use, this application helps screen mild or more severe hearing impairment by considering factors affecting the measurement results.The drawback that limits the use of this application is that it is paid and only available on the Apple iPad (21).
The AudCal TM application has excellent validity and reliability and is the last application validated with the iOS operating system.Only one study proved its validity and reliability.The average hearing threshold difference between AudCal TM and conventional PTAs was 0.21 ± 6.38 dB; however, the study did not explain whether the difference was significant or not.The kappa Cohen analysis was used to analyze the validity of AudCal TM .The AudCal TM kappa Cohen value was found to be very good, indicating that the hearing threshold match between the AudCal TM application, and the conventional audiometer in assessing hearing function was perfect.The test and retest reliability of this application were excellent.In this study, the validity and reliability of the AudCal TM application were excellent; therefore, it can be used as a screening tool to assess mild to severe hearing loss (>20 dB).However, further research is needed to increase the knowledge and confidence in its use (23).
Another application with a validated Android operating system is Hearing Test TM e-audiology, which can be downloaded for free from the Google Play Store.The Hearing Test TM eaudiology application can screen for mild hearing loss.Four studies that validated the eaudiologic Hearing Test TM application found excellent compatibility with conventional audiometry in assessing hearing function.Although its consistency with standard audiometry in assessing hearing function revealed perfect validity, the Hearing Test TM eaudiologic application still had a significant difference in the threshold obtained (14,(24)(25)(26).
The Hearing Test TM e-audiology application found remarkable differences at frequencies of 500, 1,000, 2000, and 6,000 Hz, but not at 250, 3,000, 4,000, Hz.This significant difference, particularly at 500 and 1,000 Hz frequencies, was suspected to be due to the noise around the test.No significant difference was observed at 250, 3,000, 4,000, and 8,000 Hz frequencies.However, the hearing threshold of smartphone-based audiometry was better than that of conventional audiometry in the right ear.This could be because users can drive it to any result they want due to self-administration (14).Other studies found a significant difference between Hearing Test TM e-audiologia and PTA at frequencies of 250, 500, 1.000, 2.000, 4.000, and 8.000 Hz (25,26).This resulted from a noisy testing environment in a quiet room (25).In contrast to three previous studies conducted in quiet rooms, this application demonstrated excellent validity in a soundproof room.It not only had an excellent intraclass correlation coefficient (0.85) but also a good sensitivity and specificity (99% and 79%, respectively).The application's consistency in assessing the patient's hearing threshold (reliability of the instrument) is promising (Cronbach's alpha value = 0.92), with a slight difference in the average test and retest threshold of 0.1 dB.Based on the explanation provided in the four publications above, the Hearing Test TM eaudiology application can be used as a screening tool for mild to severe hearing loss (>25-50 dB) by considering the factors that the measurement results (24).
Hearing Test Pro TM e-audiologia is another validated audiometry app for Android.In this systematic review, one study appropriately verified the eligibility of the Hearing Test Pro TM e-audiologia application.They explained that the Hearing Test Pro TM e-audiologia application has low validity and thus cannot be used as a screening tool for hearing loss.The drawback of this application is that it costs money to access, which limits its use.Its validity is determined by the type and installation of the transducer and calibration issues.Nonetheless, studies on this application are still minimal; thus, further studies are required to prove the application's validity (27).
The Wulira TM application is the final application on the validated Android system with good validity; however, due to a few limitations, the validity of the application was less than optimal.Its limitations were poor sound attenuation by the earbuds and a noisy testing environment.The excellent validity of the WuliraTM application and its similarity to conventional audiometry explain why it is suitable for use as a screening tool for mild hearing loss.However, this application was tested in a soundproof room and will undoubtedly be implemented independently by the user in a quiet (not soundproof) environment.Thus, further research is required to test its validity in different environmental conditions to increase the knowledge and evidence of its reliability Advances in smartphone technology have resulted in a plethora of healthcare applications, many of which can be self-administered by trained professionals and nonprofessionals to assess clinical populations and screen at risk individuals in areas where conventional audiometry is inaccessible.Many hearing test apps are free or inexpensive and are available to the public worldwide, including in Indonesia.They are easily accessible on smartphones running iOS (Apple) or Google Play (Android) operating systems.Self-administered hearing tests can be helpful for individuals who do not have access to auditory services or for monitoring their hearing status (19,23,29).
Research on the validity of smartphone-based audiometry applications is interesting and valuable for future research or development through primary studies.Numerous previously conducted primary studies confirm this.A previous systematic review found ten studies discussing the validation of five audiometric applications worldwide (6).As part of this systematic review, the authors found several studies that tested the validity of smartphone-based PTA applications.In this review, we searched for available applications in Indonesia, and 17 publications discussed six smartphonebased PTA applications.The 17 publications were downloaded and reviewed to verify the six smartphone-based audiometry applications by comparing them with the conventional PTA, the gold standard for evaluating hearing function (Table 1) (7,(14)(15)(16)(17)(18)(19)(20)(21)(22)(23)(24)(25)(26)(27)(28)(29).
Most studies have revealed that smartphonebased audiometry is reliable for screening hearing loss.Several factors strongly influence the advantages and disadvantages of audiometry application in assessing hearing function.These factors could not be adequately controlled based on the standard conventional audiometry.These factors include test environment noise (background noise), transducer type and method of use, calibration implementation, the digitalto-analog converter of the device, the ability to operate a smartphone, the user's attention when testing the application, the presence of cerumen in the ear canal, and user mistakes in application testing (7,(14)(15)(16)(17)(18)(19)(20)(21)(22)(23)(24)28,29).
The noise of the test environment (background noise) is a very influential factor constantly mentioned in the literature affecting the results of smartphone-based audiometric hearing function testing.The higher the environmental noise level, the worse the test results or the results of the test person's hearing threshold.It will increase the application's sensitivity while decreasing its specificity in detecting hearing loss.When performed in a soundproof booth, the application will produce a hearing threshold close to the threshold obtained with conventional audiometry; however, the room cannot be used in everyday life for independent use.Nonetheless, the room adjustment to the noise of the test environment (background noise) should be kept to a minimum (6,7,15,16,19,20,22,23).
The type of transducer and how to use a suitable transducer to minimize environmental noise entering the participant's ear are other factors affecting the application validity and are still related to environmental noise.Standard audiometric headphones are used in conventional audiometry.In contrast, most iOS application validation studies used Apple earbuds in their research because of their compatibility with iOS smartphone devices and ease of availability (7,15,16,19,21,23).Earbuds are theoretically recommended as an effective way to reduce background noise and placed in the ear canal to provide approximately 30-40 dB of noise attenuation.The placement of the earbuds is also crucial because it affects the accuracy of the measurement results, especially at low frequencies.Earphones/earbuds and circumaural earmuffs are ideally used while monitoring the test noise level.Although this is ideal, it contradicts the screening principle of using an application with minimal hardware to ensure the ease of accessibility of the smartphone-based audiometry application (7).
In addition to considering environmental noise and transducers when using smartphone-based audiometry, the application and tool calibration should be implemented.Calibration is an essential step in any testing procedure.It helps ensure equipment output and varies smartphone and transducer combinations.A certified electroacoustic laboratory continuously calibrates conventional audiometry before use (21).Although performing calibration is challenging in a standard simple calibration methods can be used (15,21,23,24).
uHear TM was calibrated before starting each test.Calibration is performed by asking the patients to enter a silent environment and then instructing them on the proper placement of the headphones or earbuds in the ear.The patient is then instructed to reduce the device volume to 50%.Finally, on the app, the patient selects the type of listening device to be used to complete the test (earbuds or headphones) (15).Another method is attaching the sound level meter to a headphone and setting the app's volume to 60 dB for 1 kHz.The app was then used to adjust the headphones' output until a 60 dB(A) reading was obtained using the sound level meter.This procedure was then repeated for each of the six audiometric frequencies used in this study.These settings were saved in the form of a calibration file (21).In contrast, the biological method involves determining the reference sound level concerning a normal hearing person's hearing threshold.Multiple calibrations on different mobile sets of the same model allow the determination of a reliable, model-specific reference sound level (23,24).
Other factors that affect the application's validity include the 16-bit digital-to-analog converter in the iPod/iPhone device, which limits the uHearTM application's dynamic range to approximately 85 (15-100) dB.Because the lower dynamic range limit (0-25 dB) is within the average hearing threshold, uHear TM may be less specific at lower ranges.Therefore, uHear TM unsurprisingly overestimates the threshold in ears with normal hearing, particularly at lower frequencies, compared with conventional audiometry, even though the application testing is performed in a soundproof room.As a result, the uHear TM application is better used in screening for impaired function, particularly in moderate to severe hearing loss (15,19).
The results of conventional audiometry and the Audiogram Mobile TM application, both tested in a soundproof room, differed.The participants were disturbed during the audiometry application test because the examiner who operated the application was also in the same soundproof booth.More distractions occurred because the participants had to pay attention to the examiner's voice while operating the iPad.This is certainly different from the test setup used for conventional audiometers, in which the examiner is outside the soundproof booth.Nonocclusive cerumen can change the acoustics of the ear canal and the overall sound pressure in the ear canal at various high frequencies.Because smartphone-based audiometric application testing should be performed independently, users should ensure that the ear canal is clean before the test (21).
User error is another factor that affects the validity of an audiometric application while testing the application.Users can direct it to any result they want because, unlike conventional audiometry, audiologists perform the test administration (14).Using audiometry applications supervised by audiologists is preferred.This ensures no user errors, such as eliminating specific testing frequencies, can occur by accidentally pressing buttons or switching sides of the headphones and monitoring environmental noise (24).Users must understand the application instructions before avoiding user errors when operating the application (20).
Another factor affecting the validity of smartphone-based audiometry is that the older population may be unable to operate a smartphone because they are not actively using one.This factor contributes to poor test results because the older population is frequently overlooked when tapping the screen (16)(17)(18).Conventional audiometry is the gold standard of hearing testing performed by audiologists in appropriate facilities.Because it is not available in every primary health facility because of financial constraints, most patients are referred to better health facilities for diagnostic audiometry examinations by primary health doctors (19,22).This makes it challenging to evaluate many people using conventional audiometry.To overcome this, a new hearing function assessment method, smartphone-based audiometry, has been widely developed as a hearing function screening method to provide patients with easy access and reliable results.The low cost of testing with smartphone-based audiometry and its simplicity and ease of use make it preferable to conventional audiometry testing, even though it is only limited to screening.Smartphone-based screening has great potential due to the use of smartphones and cellular networks globally (14,29).
Most of the literature in this review reported that smartphone-based audiometry was still inferior to conventional audiometry for screening hearing loss.This difference was found to be due to many factors affecting the test results in smartphone-based audiometry applications, which are difficult to control properly.The influencing factors include the noise of the test environment (background noise), the type and method of the transducer, the implementation of calibration, the digital-toanalog converter of the device, the ability to operate a smartphone, the user's attention when testing the application, the presence of cerumen in the ear canal, and user errors in testing.Furthermore, all reviewed studies in this systematic review demonstrate that smartphonebased audiometry is not intended to detect hearing loss more precisely or specifically.Because smartphone-based audiometry applications only test the air conduction hearing threshold using a transducer in the form of paired headphones to cover the outer ear, determining whether a person has conductive, sensorineural, or mixed hearing loss is impossible (7,(14)(15)(16)(17)(18)(19)(20)(21)(22)(23)(24)(25)(26)(27)(28)(29).

Conclusion
Smartphone-based audiometry can be used as a screening modality or an alternative to assess hearing function.uHear TM , Audiogram Mobile TM , AudCal TM , Hearing Test TM eaudiologia, and Wulira TM applications that have been validated or have good validity are recommended for use in areas without access to conventional audiometric examinations, primary health services, and independent use in Indonesia.Smartphone-based audiometry is inexpensive, simple, and more accessible than conventional audiometric testing.Smartphonebased audiometry cannot replace conventional audiometry in assessing hearing function because it can only measure the air conduction threshold and thus cannot determine hearing loss.Furthermore, its validity is not as good as that of conventional audiometry.Several factors that influence the measurement results, such as conventional audiometry, are uncontrollable.

Fig 1 :
Fig 1: Results of the literature search with PRISMA stages.